FEAT Plug-in mechanism for private scenarios by ValbuenaVC · Pull Request #2131 · microsoft/PyRIT

ValbuenaVC · 2026-07-08T00:53:32Z

Adding Private Scenario Support via Plug-Ins

Goal

Let an operator activate private scenarios and attack techniques from a stock PyRIT
installation — without forking PyRIT or passing per-run CLI flags — so a team's private
red-teaming package behaves like a built-in in the standard backend, GUI, and pyrit_scan
catalog.

# .pyrit_conf
plugins:
  - name: my_redteam
    source: /repos/my-redteam
    initializer: my_redteam.setup.MyInitializer

# A private technique through a public scenario
pyrit_scan airt.rapid_response --target openai_chat --techniques operation_foobar
# A private scenario the initializer registers
pyrit_scan my_redteam.violence --target openai_chat

What a plug-in is

A plug-in is a config-declared pointer to a private PyRITInitializer that lives outside
the PyRIT tree. It is one thin layer above --initialization-scripts:

config-declared (backend/GUI honor it; no per-run flag);
run first (privileged), before other initializers and before catalog/metadata
warming, so lazy consumers see a complete registry;
package-root-anchored — source is placed on sys.path, so a packaged initializer
with intra-package imports (e.g. from my_redteam... import) loads correctly, which a
loose --initialization-scripts path cannot do.

PluginSpec is {name, source, initializer}:

name — an operator label used in logs/errors;
source — the folder that contains your package, placed on sys.path (relative resolves
against the config file);
initializer — a dotted module.Class path to a concrete PyRITInitializer.

ConfigurationLoader prepends this as a privileged initializer automatically; operators
cannot list it under initializers: (it is not a registered initializer name).

Why not just order the config?

pyrit_scan is a thin REST client, and the backend resolves a scenario before it runs
any per-request initializers — so "list your important initializer first" is too late for
scanner discovery. The backend also initializes once at startup from its config file, not
per client command. And initializers: only resolves built-in registered names, while
initialization_scripts: loads a loose file without anchoring a package root. The plug-in
covers exactly this gap: startup-config-driven, guaranteed-first, and package-root-anchored.

The initializer owns registration

PyRIT discovers nothing on its own. The plug-in's initializer registers whatever it wants
discoverable, at whatever level of abstraction the private content requires:

techniques via AttackTechniqueRegistry.register_from_factories(...) → --techniques;
scenarios via ScenarioRegistry.register_class(cls, name=...) → pyrit_scan <name>;
datasets via provider registration + memory load (private seeds stay in the
operator's database, never published);
default targets via set_default_value(...).

This is deliberately flexible: "private" is rarely all-or-nothing. Sometimes only a
dataset must stay private, sometimes a niche technique, sometimes an entire scenario
should not be exposed at all — the initializer publishes the generic parts and registers
only the sensitive ones.

Behavior

Fail-closed: any failure (missing source, unresolvable/non-PyRITInitializer target,
or an initializer that raises) aborts initialization with a precise error. A restart is
required after fixing the config or source.
Trust boundary: loading runs third-party Python with backend permissions. Whoever can
write the config or the source can run code on the host. No transactional rollback of
arbitrary side effects is claimed, and dependencies must already be installed in the
backend environment.
One plug-in per config.

Notable changes

Restores RapidResponse to expose all registered techniques (drops a core-only
filter, aligning with FIX: Aligning Scenarios with Technique Registry #2186), so an initializer-registered private technique surfaces via
--techniques with no scenario-routing tag.
Removes the earlier discovery-based machinery: source/scenario/technique discovery, the
scenario:<name> technique_tags routing bridge, wheel extraction/versioning, the
transactional registry snapshot, and the contributed/external registration APIs.

Testing

Full unit suite: 10,417 passed, 120 skipped.
Gated plug-in integration (RUN_ALL_TESTS=true): a source package whose initializer
registers a real private scenario + technique is discoverable; a bad initializer fails
closed.
ty check pyrit: clean. Full pre-commit run --all-files: all hooks pass.

Limitations / out of scope

Only one plug-in; no composition.
No component discovery, no scenario: tags, no wheels/builder, no version resolution,
no dataset/target/scorer/converter plug-in types (the initializer registers these).
No hot reload; activation is at process/backend startup only.
Built-in name shadowing is accepted (the trusted initializer owns its namespace); a
shadow-warning is a possible follow-up.
A private package must register its scenarios in its initializer and be on the current
ScenarioTechnique API before it will load; PyRIT does no runtime API repair.

Adds a plug-in mechanism so an operator can run non-disclosable scenarios (and self-contained datasets) from a stock public PyRIT install, from a pre-built wheel referenced by PLUGIN_WHEEL, without those components living in the public repo. Plug-in loading runs as a guaranteed-first phase inside initialize_pyrit_async: after central memory is set and before the configured initializers, so plug-in datasets/scenarios register before LoadDefaultDatasets and PreloadScenarioMetadata read the registry. Ordering is true by construction, independent of .pyrit_conf list position. The wheel is extracted (stdlib zipfile - never pip/.venv) to .plugin/<name>/, prepended to sys.path, imported (dataset providers self-register), and its bootstrap (a top-level register() callable or a shipped PyRITInitializer subclass) is run; the loader then asserts the plug-in registered something. No-op when PLUGIN_WHEEL is unset. Fails closed by default; fail-open via PLUGIN_FAIL_OPEN or initialize_pyrit_async(plugin_fail_open=True). Guards against silent-failure modes: extraction (not zipimport) so __file__-relative datasets resolve, atomic extraction, submodule discovery so components register even if the package __init__ does not import them, a package-shadowing guard when an installed package of the same name is importable, a loud warning when a plug-in dataset name collides with an existing name (the resolver is memory-authoritative, so the plug-in copy would otherwise be silently ignored), and rollback of sys.path / sys.modules / registries on failure. Wiring: .gitignore keeps .plugin/ (ignores its contents); .env_example documents the PLUGIN_* variables. Tests build a mock wheel (no dependency on any real plug-in) covering extraction, registration, ordering, collisions, and failure modes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Adds a wheel-based plug-in loading mechanism to PyRIT initialization so operators can ship private scenarios (and dataset providers) outside the public repo, while ensuring plug-ins load before other initializers consume registries.

Changes:

Introduces pyrit.setup.plugin_loader to extract a configured .whl into .plugin/, import it, run a bootstrap (register() or PyRITInitializer), and validate registration with rollback-on-failure.
Hooks plug-in loading into initialize_pyrit_async as a guaranteed-first phase (after memory is set, before other initializers), with a plugin_fail_open override.
Adds extensive unit coverage for loader behavior/guards and documents operator configuration via .env_example and .pyrit_conf_example; ensures .plugin/ is kept but ignored in git.

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`tests/unit/setup/test_plugin_loader.py`	New unit test suite that builds a mock wheel and exercises loader spine + guardrails (fail-open/closed, rollback, collisions, cache, package inference).
`pyrit/setup/plugin_loader.py`	New plug-in loader implementation: extraction, import/submodule discovery, bootstrap execution, collision warnings, and rollback.
`pyrit/setup/initialization.py`	Adds the plug-in load phase into `initialize_pyrit_async` and exposes `plugin_fail_open`.
`.pyrit_conf_example`	Documents that plug-ins are not configured via `.pyrit_conf` and are loaded automatically during initialization.
`.plugin/.gitkeep`	Keeps `.plugin/` directory present in the repo while allowing extracted artifacts to be ignored.
`.gitignore`	Ignores extracted plug-in artifacts under `.plugin/` while retaining `.gitkeep`.
`.env_example`	Documents `PLUGIN_WHEEL`, `PLUGIN_PACKAGE`, `PLUGIN_FAIL_OPEN`, and `PLUGIN_DIR` configuration and trust boundary.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Adds an integration test that loads a plug-in wheel through initialize_pyrit_async and asserts every scenario the wheel ships is registered in ScenarioRegistry. Two cases: a self-contained case builds a small wheel at test time and checks its scenarios are discovered (guards the mechanism in public CI); an injected case loads a wheel supplied out-of-band via PLUGIN_TEST_WHEEL (+ PLUGIN_TEST_SCENARIO_DIRS or PLUGIN_TEST_PACKAGE) and verifies all of its scenarios are picked up. The injected case is skipped unless those variables are set, so the committed test depends on no specific external package; a downstream CI job can point it at a real scenario wheel to guarantee full discovery. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Both the dataset provider registry (keyed by class name) and the scenario catalog (keyed by registry name) assign unconditionally, so a plug-in whose provider or scenario name collides with an existing one silently replaces the original on registration. Rollback previously only deleted the keys the plug-in added, leaving a collided entry permanently replaced after a failed (or fail-open) load. Rollback now restores the pre-load value for any entry the plug-in overwrote, and only deletes keys the plug-in newly added. Entries present before the load (including built-ins discovered meanwhile) are still preserved. Adds unit tests covering the provider and scenario overwrite-then-restore paths. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Replace the raw zipfile.extractall (and the hand-rolled path-traversal check) in the wheel extractor with pyrit.common.safe_extract.safe_extract_zip, the existing helper used for untrusted archive extraction. It validates every member before writing anything: path traversal, absolute/drive paths, symlink and other non-regular entry types, per-file and total size caps, entry count, and compression ratio (zip bomb), then sanitizes extracted permissions. Extraction stays atomic (temp dir then os.replace) and the cache-reuse path is unchanged. Adds a unit test asserting a wheel with a path-traversal member is rejected and nothing is written outside the extraction directory. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

_load_plugin_async is reached from the async initialization path but performed wheel extraction and directory scanning inline, which blocks the event loop and serializes unrelated async work during startup. Wrap the two filesystem-bound steps (_extract_wheel and _resolve_package_name) in asyncio.to_thread. Package import and bootstrap stay on the loop thread since they mutate global registries and sys state and represent the registration step rather than pure I/O. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

A single PluginLoadError collapsed every failure mode into one type. Add subclasses so callers can distinguish causes: PluginWheelNotFoundError (PLUGIN_WHEEL missing or not a .whl), PluginImportError (package import failed, incl. an installed package shadowing the wheel), and PluginRegisteredNothingError (imported but registered nothing). All subclass PluginLoadError, so existing `except PluginLoadError` handling and the fail-open path are unchanged. The load boundary preserves the specific type while still prefixing the standard "Failed to load plug-in ... <remediation>" guidance; unknown errors (a raising bootstrap, or an unsafe archive) remain a plain PluginLoadError. Tests assert the specific type per failure mode and the shared hierarchy. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

_extract_wheel used a temp dir keyed only by wheel stem + PID. Once extraction was moved off the event loop with asyncio.to_thread, two concurrent loads of the same wheel in one process could collide on that shared path, where one thread's rmtree clobbers the other's in-progress extraction (crash or corrupted cache). Allocate a unique temp dir per extraction with tempfile.mkdtemp (atomic, 0700) so concurrent extractions never share a path. Surfaced by security review of the to_thread change. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

fdubut

Also as mentioned in our side conversation, I think it would be useful to share an example of how a customer would package their datasets/scenarios to make them available as a plugin.

The prior name did not convey what the flag does. Rename the parameter, env var (PLUGIN_FAIL_OPEN -> PLUGIN_ACCEPT_LOAD_FAILURES), and internals so the intent - accept a plug-in load failure and continue - is clear at the call site. Behavior is unchanged: fail-closed by default; explicit param overrides env; env overrides default. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

…a-microsoft-plugin-mechanism

…port Move plug-in configuration out of .env and into a dedicated .pyrit_conf 'plugins' list. Each entry is a wheel path, a {package: wheel} pair, or an explicit {wheel, package} mapping, parsed into a PluginSpec. initialize_pyrit_async now takes a plugins sequence and loads them in list order as the guaranteed-first phase, so one plug-in behaves identically to several. A configless setup simply has no plug-ins. Plug-in loading stays a fixed phase (after memory is set, before configured initializers) rather than a user-ordered initializer, so 'plug-ins register before anything reads the registry' remains true by construction. A new _verify_plugin_prerequisites health check gates loading on the prior init phases (env, memory) having completed. The PLUGIN_WHEEL/PLUGIN_PACKAGE env vars are removed as config sources; PLUGIN_DIR and PLUGIN_ACCEPT_LOAD_FAILURES remain env-overridable. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

…a-microsoft-plugin-mechanism

Add ScenarioRegistry.register_external_subclasses to register a plug-in package's concrete Scenario subclasses by type (without triggering built-in discovery), refactor the shared _register_subclasses_in_package helper out of _discover, and add an overridable _external_registry_name hook. ScenarioRegistry keys plug-in scenarios by their module path relative to the plug-in package, mirroring built-in dotted names so image/text classes with the same leaf name do not collide. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Add pyrit/setup/plugin_compat.py with best-effort drift shims: warn_on_version_drift (reads the plug-in's Requires-Dist: pyrit pin and warns loudly on a minor mismatch) and bridge_scenario_extension_points (makes a plug-in Scenario concrete by bridging a renamed async extension point when the class is abstract solely due to that rename). Wire both into PluginLoader._load_plugin_async, and after bootstrap call ScenarioRegistry.register_external_subclasses so scenario-only plug-ins register without an explicit bootstrap. Add a size+mtime extraction-cache fingerprint (.plugin_wheel_fingerprint) so a rebuilt wheel with an unchanged filename re-extracts instead of reusing a stale extraction. Warnings are loud and greppable (PLUGIN VERSION DRIFT / PLUGIN COMPAT SHIM / PLUGIN CACHE STALE); irreconcilable drift still fails closed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

… and execute Unit: add two tests that a plug-in's Scenario subclass is auto-registered by type-scoped discovery without a bootstrap, and that a bootstrap-registered scenario is not duplicated by auto-registration. Integration: refactor the injected-wheel test into fixtures (build_mock_wheel, plugin_dir, all_scenarios) and add coverage that every scenario in an out-of-band wheel is discovered and instantiates, plus an env-gated case (PLUGIN_TEST_EXEC_SCENARIO + ADVERSARIAL_CHAT_ENDPOINT) that drives one scenario through the public set_params_from_args/initialize/run pipeline. The execution case tolerates a live-endpoint 'objectives incomplete' outcome (the attack still ran) but re-raises any other exception so a real regression fails loudly. The always-on mock case stays generic with no external package named. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

rlundeen2 · 2026-07-09T22:54:17Z

+
+from __future__ import annotations
+
+import inspect


I don't dislike this solution, but I'm worried it's more complex than we need to solve the problem we're tackling.

I'd recommend seeing if this makes our lives simpler or if you see it making our lives simpler going forward. My gut reaction is it will be easier to just include internal

technique initializers

datasets

the pyrit_scan to run

I've spent some time refactoring this and will have the changes in soon for additional review. A few notes:

I want to scope this PR for just private scenarios and attack techniques. The groundwork will be there for other components, but I think they introduce enough complexity (especially datasets) that they deserve their own PRs.

Earlier I conflated user discoverability with our actual implementation. Expecting users to build a well-formatted Python wheel for their plug-in is great for scaling plug-ins long term, but bad for live ops usage, even with a follow-up PR for a pyrit wheel builder. I think operators should be able to just write a config that includes the path to their current operation's scenarios/techniques and have PyRIT discover them.

For simplicity's sake the config should be the source of truth for plug-ins. I don't think for this PR pyrit_scan should take any responsibility over plugins. The ConfigurationLoader and initialize_from_config_async components should handle them specifically because they can force precedence for plug-ins prior to anything else. Ideally pyrit_scan should remain very thin, and assume that by the time _dispatch_with_client_async is called, the plug-in components have already been added to their relevant registries.

Move the plug-in load error hierarchy into pyrit.exceptions and re-export it from the package while preserving the loader's existing exception identities and behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Add source, discovery, validation, and collision errors under PluginLoadError and align the empty-contribution error with the V1 scenario/attack-technique scope. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Introduce the explicit V1 PluginSpec schema, resolve artifacts relative to their config file, reject composition, and remove the fail-open config surface while preserving the existing wheel loader behind the normalized spec. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Have ConfigurationLoader inject a framework-owned PluginInitializer ahead of user initializers, remove the special plug-in phase and fail-open API from core initialization, and enforce the single-plugin fail-closed V1 lifecycle. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Introduce the common PreparedPlugin artifact model and WheelPluginFormat adapter, route the loader through safe threaded preparation, retain advisory version metadata, and remove duplicated extraction/package-resolution logic. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Add a non-executing source adapter for one importable Python file with canonical ownership metadata and content fingerprinting behind the common PreparedPlugin contract. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Extend source format preparation to proper Python packages, validate nested entry modules and ownership, reject loose directories, and fingerprint package trees deterministically. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Import prepared source or wheel modules through a shared ownership-checked path and discover concrete module-owned Scenario subclasses with deterministic source-relative dotted names and ambiguity errors. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Discover explicit, conventional, and module-global AttackTechniqueFactory contributions without invoking arbitrary helpers, require scenario applicability, validate identifiers, and reject duplicate technique names. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Replace plug-in-authored bootstrap behavior with framework-owned source/wheel discovery, strict extend-only scenario and technique registrars, latent built-in collision checks, unsupported provider cleanup, and fail-closed registry/module rollback. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Store scenario applicability on contributed factories and have RapidResponse union its built-in core catalog with private techniques explicitly targeting airt.rapid_response, without requiring private factories to claim the core tag. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Migrate wheel integration to the ConfigurationLoader-owned privileged path and complete source scenario and RapidResponse technique tests through a freshly launched stock pyrit_scan backend. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Document when to use config-driven plug-ins instead of direct Python composition, add source/wheel setup and troubleshooting guides, update .pyrit_conf examples, and surface the guidance in pyrit_scan --help without adding scanner activation logic. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Seed the privileged core technique prerequisite, validate contributed scenario metadata transactionally before startup completes, remove runtime API mutation shims, and align wheel tests with framework-owned lifecycle behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Use a side-effect-light scenario with inline objective and scorer so plug-in metadata validation is deterministic in isolated and parallel test processes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Copilot

Pull request overview

Copilot reviewed 34 out of 35 changed files in this pull request and generated 3 comments.

+        plugins: List of plug-ins to load as the guaranteed-first initialization phase. Each
+            entry is a wheel path string, a ``{package: wheel}`` mapping, or a
+            ``{wheel: ..., package: ...}`` mapping. Empty means no plug-ins.


+# Keeps the .plugin/ directory in version control while its contents stay ignored.
+# PyRIT extracts plug-in wheels here at runtime (see PLUGIN_WHEEL).


+            raise PluginDiscoveryError(
+                f"Plug-in module '{module.__name__}' defines multiple Scenario classes. "
+                "Provide explicit registry names through the plug-in manifest."
+            )


Bring the feature branch forward by 21 upstream commits while preserving source and wheel plug-in activation. Resolve the ScenarioStrategy-to-ScenarioTechnique conflict by adopting the new technique API, retaining scenario-specific contributed-factory selection in RapidResponse, updating plug-in examples and tests, and refreshing collision fixtures for the reorganized role-play catalog. Address automated review feedback by documenting the explicit PluginSpec schema, correcting the PLUGIN_DIR extraction override, and replacing the unsupported manifest remediation with separate-module guidance. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

The plug-in PR had RapidResponse filter its technique pool to the built-in 'core' tag via get_factories_for_scenario(base_query=TagQuery.all('core')), with a fallback whose return value was discarded. Restore the builder to the upstream form (aligning with microsoft#2186): read every registered factory via get_factories_or_raise and build the technique enum with no core filter. This lets an initializer-registered private technique (any tag) surface through --techniques with no scenario-routing bridge, and removes a latent empty-enum path. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Remove the registration surface the old discovery-based plug-in system needed, now that a plug-in points at a PyRITInitializer that registers components itself via the ordinary register_from_factories / register_class paths: - AttackTechniqueRegistry.register_contributed_factory and get_factories_for_scenario (the scenario:<name> tag bridge consumer; no remaining callers after the RapidResponse restore). - ScenarioRegistry.register_contributed_scenario and _external_registry_name. - Registry.register_external_subclasses, _external_registry_name, and the external_package branch of _register_subclasses_in_package (dead code with no callers). V1 is intentionally extend-only-by-trust: the contributed-* methods were the only built-in name-collision guards, so a plug-in initializer may now shadow a built-in name. This is accepted for the trusted-artifact model and revisited post-MVP. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

… and wheels Reframe a plug-in as a config-declared pointer to a private PyRITInitializer that lives outside the PyRIT tree, run privileged-first. The framework only anchors the source root on sys.path, imports the dotted initializer, runs it, and fails closed; the initializer owns all registration (scenarios, techniques, datasets, targets). - PluginSpec is now {name, source, initializer} (source-only; no format/wheel/package). - PluginInitializer resolves and runs the dotted initializer; no component discovery, no transactional snapshot/rollback, no version-drift machinery. - Delete plugin_discovery.py and plugin_formats.py (wheel extraction, source/scenario discovery, the scenario: tag bridge). - Prune plug-in exceptions to PluginLoadError + PluginSourceNotFoundError. - Drop the PluginFormat export and refresh the ConfigurationLoader plugins docstring. This is a thin convenience + ordering layer over --initialization-scripts: config-declared, run-first, and package-root-anchored so a packaged initializer with intra-package imports loads. Wheels, discovery, and versioning are out of scope for the MVP. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Remove wheel/discovery/scenario-tag content and describe the source-only, initializer-pointing plug-in end to end. - .pyrit_conf_example and doc/getting_started/pyrit_conf.md: {name, source, initializer} schema. - doc/getting_started/plugins.md: initializer contract, the plug-in-vs-initialization-scripts decision, and how the initializer registers scenarios/techniques/datasets/targets (including gray-area content at different abstraction levels). - doc/getting_started/troubleshooting/plugins.md: source-path, import, and non-initializer failures. - Remove the .plugin/ extraction dir and its .gitignore/.gitkeep entries; drop a stray .env_example blank line. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Replace the discovery/wheel/contributed-registration test surface with tests for the initializer-pointing model. - test_plugin_spec.py: the {name, source, initializer} schema and its validation. - test_plugin_loader.py: PluginInitializer anchors the source root, runs the dotted initializer, and fails closed on a missing source, an unresolvable/non-PyRITInitializer target, or an initializer that raises. - test_plugin_loader_integration.py: a source package whose initializer registers a real private scenario + technique becomes discoverable (gated). - Update test_configuration_loader plug-in cases to the new schema. - Delete test_plugin_discovery/test_plugin_formats/test_plugin_registration. - Drop the contributed-technique cases from test_rapid_response and test_attack_technique_registry, and the removed-exception cases from test_exceptions. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

Resolve the inline review TODOs and genericize the plug-in docs/config for a shipped feature: - Explain, in plain language, what 'source' should point at and why the package root goes on sys.path (and the consequences of getting it wrong). - Replace the 'do not list it under initializers:' instruction with what actually happens: the privileged initializer is not a registered name, so listing it fails; the framework injects it. - Reframe the 'gray-area' guidance from the operator's perspective (contribute publicly vs keep tracked vs keep private; sometimes only the dataset/technique/scenario is sensitive). - Remove all 'V1' version references from docs, config example, and code messages/docstrings. - Replace every pyrit-internal / RapidResponseInitializer example with a generic my_redteam package, and rename the integration-test namespace accordingly. - Fix a stale 'source or wheel' docstring left in ConfigurationLoader._normalize_plugins. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 153315ea-4360-489a-89c5-630a41cf3fd2

ValbuenaVC requested review from Copilot, fdubut and romanlutz July 8, 2026 01:00

Merge branch 'main' into vvalbuena-microsoft-plugin-mechanism

c7a7528

Copilot started reviewing on behalf of ValbuenaVC July 8, 2026 01:00 View session

Copilot AI reviewed Jul 8, 2026

View reviewed changes

Comment thread pyrit/setup/plugin_loader.py Outdated

Comment thread pyrit/setup/plugin_loader.py Outdated

Comment thread pyrit/setup/plugin_loader.py Outdated

Comment thread pyrit/setup/plugin_loader.py Outdated

ValbuenaVC and others added 3 commits July 8, 2026 09:52

Merge branch 'main' into vvalbuena-microsoft-plugin-mechanism

e7d63cf

Potential fix for pull request finding

eca5928

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Merge branch 'main' into vvalbuena-microsoft-plugin-mechanism

0b11b6c

ValbuenaVC changed the title ~~[DRAFT] FEAT Plug-in mechanism for private scenarios~~ FEAT Plug-in mechanism for private scenarios Jul 8, 2026

ValbuenaVC and others added 7 commits July 8, 2026 10:49

Merge branch 'main' into vvalbuena-microsoft-plugin-mechanism

8781038

fdubut reviewed Jul 8, 2026

View reviewed changes

Comment thread pyrit/setup/initialization.py Outdated

Comment thread .env_example

Comment thread .env_example Outdated

Comment thread pyrit/setup/plugin_loader.py Outdated

ValbuenaVC and others added 11 commits July 9, 2026 09:38

Merge branch 'main' into vvalbuena-microsoft-plugin-mechanism

b551008

Merge commit 'b55100804be2f4fae26d619cbd5b072e104a94ce' into vvalbuen…

67f8a85

…a-microsoft-plugin-mechanism

Merge branch 'main' into vvalbuena-microsoft-plugin-mechanism

067ffba

Merge commit '067ffba6461c6e89590a3d369a3d5bcc408508a3' into vvalbuen…

b21f856

…a-microsoft-plugin-mechanism

Merge branch 'main' into vvalbuena-microsoft-plugin-mechanism

21ebf36

Merge commit '21ebf36815e93891ef55221912fee2345bd311c9' into vvalbuen…

466679a

…a-microsoft-plugin-mechanism

rlundeen2 reviewed Jul 9, 2026

View reviewed changes

Victor Valbuena added 15 commits July 13, 2026 15:53

ValbuenaVC requested a review from Copilot July 14, 2026 17:04

Copilot started reviewing on behalf of ValbuenaVC July 14, 2026 17:05 View session

Copilot AI reviewed Jul 14, 2026

View reviewed changes

ValbuenaVC requested a review from varunj-msft July 14, 2026 17:57

ValbuenaVC and others added 9 commits July 14, 2026 12:13

Merge branch 'main' into vvalbuena-microsoft-plugin-mechanism

f7ea23d

Merge branch 'main' into vvalbuena-microsoft-plugin-mechanism

7bc560b

Merge branch 'main' into vvalbuena-microsoft-plugin-mechanism

a7800f3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

FEAT Plug-in mechanism for private scenarios#2131

FEAT Plug-in mechanism for private scenarios#2131
ValbuenaVC wants to merge 49 commits into
microsoft:mainfrom
ValbuenaVC:vvalbuena-microsoft-plugin-mechanism

ValbuenaVC commented Jul 8, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fdubut left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rlundeen2 Jul 9, 2026

Uh oh!

rlundeen2 Jul 9, 2026

Uh oh!

ValbuenaVC Jul 13, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		# Keeps the .plugin/ directory in version control while its contents stay ignored.
		# PyRIT extracts plug-in wheels here at runtime (see PLUGIN_WHEEL).

Uh oh!

Conversation

ValbuenaVC commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Adding Private Scenario Support via Plug-Ins

Goal

What a plug-in is

Why not just order the config?

The initializer owns registration

Behavior

Notable changes

Testing

Limitations / out of scope

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fdubut left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rlundeen2 Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

rlundeen2 Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

ValbuenaVC Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ValbuenaVC commented Jul 8, 2026 •

edited

Loading

ValbuenaVC Jul 13, 2026 •

edited

Loading